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Abstract 

Let G be a countable discrete group. Call two subgroups Hi and H2 of G commen- 
surable if Hi n H2 has finite index in both Hi and H2. We say that an action of G on a 
discrete set X has noncommensurable stabilizers if the stabilizers of any two distinct points 
of X are not commensurable. We prove in this paper that the action of the mapping class 
group on the complex of curves has noncommensurable stabilizers. Following a method 
due to Burger and de la Harpe, this action leads to constructions of irreducible unitary 
representations of the mapping class group. 

Mathematics Subject Classifications: Primary 57N05; Secondary 20F38, 22D10, 22D30. 

1. Introduction 

Throughout the paper, M will denote a compact connected oriented surface, possibly 
with boundary, and P = {pi, . . . ,p ra } a finite collection of m points (called punctures) in 
the interior of M. Define TC(M, P) to be the group of orientation-preserving homeomor- 
phisms h : M — > M such that h is the identity on the boundary of M and h(P) = P. Two 
homeomorphisms h and h! in TC(M, P) are said to be isotopic if there exists a continuous 
family h t G 7i(M,P), t 6 [0, 1] such that h<$ = h and hi = h' . The mapping class group 
Ji4(M, P) is the group of isotopy classes in TC(M, P). 

Let G be a countable discrete group. An unitary representation is a homomorphism 
7r from G to the unitary group of a separable Hilbert space V. We say that n is irreducible 
if V does not contain any closed subspace invariant by the action of G, except {0} and V 
itself. The space of equivalence classes of irreducible unitary representations of G is called 
the unitary dual of G and is denoted by G. We will also denote by G id the subspace of 
G of equivalence classes of finite dimensional irreducible unitary representations. Besides 
the particular case of virtually abelian groups (see [War, Theorem 5.4.1.4]), there is no 
systematic procedure of constructing all irreducible unitary representations of G. However, 
one may hope to construct large families of them. 

Recall that two subgroups Hi and H2 of G are commensurable if Hi fl H2 has finite 
index in both Hi and H2. We say that an action of G on a discrete set X has noncommen- 
surable stabilizers if the stabilizers of any two distinct points of X are not commensurable. 
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A method to construct irreducible unitary representations is proposed by Burger and de 
la Harpe in [BH]. This method consists first in translating some results due to Mackey 
[Mac] into the terminology of the discrete group theory, and then to apply these Mackey's 
results to the situation of an action with noncommensurable stabilizers. 

With a punctured surface (M, P) one can associate a simplicial complex X{M,P) 
called complex of curves. This complex has been introduced by Harvey [Harv] and plays a 
prominent role in the study of mapping class groups (see [BLM], [Hal], [Ha2], [Ivl], [Iv2], 
[IM], [McC], [MM1], [MM2]). 

We consider in this paper X(M, P) as a set, ignoring the simplicial structure. Our 
main result is that the action of M{M, P) on X(M, P) has noncommensurable stabilizers. 
We also describe the structure of the stabilizers, leading by this way to constructions 
of irreducible unitary representations of Ai(M, P). Furthermore, this action provides 
interesting phenomena that show the limits of a possible generalization of Mackey's results. 

Our work is organized as follows. In Section 2 we give the definition of the complex 
of curves X{M, P) and study some basic examples. In Setion 3 we state the method of 
Burger and de la Harpe of constructing irreducible unitary representations. In Section 4 
we state some preliminary results on curves and Dehn twists which will be useful for the 
remainder of the paper. These preliminary results are enterely proved in [PR2]. Section 
5 is dedicated to our main result and its consequences, and we study in Section 6 the 
structure of the stabilizers. 



2. Definitions and examples 

By a simple closed curve in M \ P we mean an embedding a : 5" 1 — > M \ P. Note that 
a has an orientation; the curve with opposite orientation but same image will be denoted 
by a -1 . By abuse of notation, we also use a for the image of a. We say that a is generic if 
it does not bound a disk in M containing zero or one puncture. Two simple closed curves 
a, 6 : S 1 — > M\P are isotopic if there exists a continuous family at : S 1 — > M\P, t G [0, 1] 
of simple closed curves such that ao = a and a\ = b. Isotopy of curves is an equivalence 
relation which we denote by a ~ b. Note that h(a) ~ h'(b) if a ~ b and if h, h' G H(M, P) 
are isotopic. So, the mapping class group M(M,P) acts on the set of isotopy classes of 
simple closed curves. 

Define a generic r -family of disjoint curves to be a r-tuple (ai, . . . , a r ) of generic closed 
curves such that: 

(a) ai fl Oj = 0, for i ^ j; 

(b) ai is neither isotopic to aj nor a" 1 , for i ^ j; 

(c) ai is not isotopic with a boundary curve of M. 

We say that two generic r- families of disjoint curves A = (a\, . . . , a r ) and B = (b\, . . . ,b r ) 
are equivalent if there exists a permutation a G E r such that ~ for each i = 1, . . . ,r. 
We will write [ai, . . . , a r ] for the equivalence class of (ai, . . . , a r ). Then X r (M, P) denotes 
the set of equivalence classes of generic r-families of disjoint curves. Notice that Xl(M, P) 
is the set of isotopy classes of nonoriented generic curves. The action of Ai(M, P) on 



2 



X r (M,P) is induced by the action of M.(M,P) on the set of isotopy classes of simple 
closed curves. Now, define the complex of curves to be: 

oo 

X(M,P)= (Jx r (M,P) . 

r=l 

Example 1. M(S 2 ) = M(S 2 ,{p}) = M(D 2 ) = M(D 2 ,{p}) = {1}. These are the only 
mapping class groups which are trivial. A4(S 2 , {pi,P2}) is a cyclic group of order two, and 
M(S 2 , {pi,P2,P3}) is isomorphic with the group S 3 of permutations of {1, 2, 3}. These are 
the only non-trivial mapping class groups which are finite. Note that, in these examples, 
X(M, P) is empty. However, the group being finite, the irreducible unitary representations 
of AA(M, P) are well understood. 

Example 2. There are four abelian mapping class groups. The first one is the mapping 
class group of the cylinder: ^(S 1 x I) = Z. The other three cases are of importance 
for the remainder of the paper. Choose two points pi,P2 in the interior of the disk D 2 
and call a pair equivalent to (-D 2 , {^1,^2}) a pantalon of type I. Choose a point p in the 
interior of the cylinder S 1 x I and call a pair equivalent to (S 1 x i", {p}) a pantalon of type 
II. Call a pair equivalent to (M, 0), where M is the sphere with three holes, a pantalon of 
type III. M. (-D 2 , {pi, P2}) is the infinite cyclic group generated by the "half-twist" which 
interchanges the punctures. M-^S 1 x /, {p}) is isomorphic with Z 2 and is generated by the 
Dehn twists along the boundary components. Let M be the sphere with three holes. M.{M) 
is isomorphic with Z 3 and is generated by the Dehn twists along the boundary components. 
Like in the finite case, X(M, P) is empty in these four examples. Nevertheless, the group 
being abelian, the irreducible representations of Ai(M, P) are well understood. Note that 
there is no other example where X(M, P) is empty. Moreover, if A4(M,P) is virtually 
abelian, then (M, P) is as in Example 1 or 2, thus either M(M 7 P) is finite or free abelian. 

Example 3. The mapping class group of the torus T 2 = S 1 x S 1 with either zero or 
one punture is the modular group of 2 x 2 matrices with integer coefficients: Ai(T 2 ) = 
M(T 2 , {p}) = SL(2, Z). There is only one M{T 2 ) -orbit in X(T 2 ) represented by [a], where 
a = S 1 x *. The stabilizer of [a] is the subgroup of upper triangular matrices in SL(2, Z) 
(with diagonal entries ±1) which is known to be its own commensurator in SL(2, Z) by 
[BH]. 




Figure 1: Some curves in the closed surface of genus 2 

Example 4. Assume that M is a closed surface of genus 2, and let 01,02,03,04 be the 
simple closed curves drawn in Figure 1. There are 6 7W(M)-orbits in X{M) represented 
by [ai], [a 4 ], [ai,a 2 ], [ai,a 4 ], [ai,a 2 ,a 3 ], [ai,a 3 ,a 4 ]. 
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Example 5. The mapping class group of the disk with m = \P\ punctures is known to be 
the Artin braid group B m on m strings. Consider a simple closed curve a : S 1 — > D 2 \ P. 
Then a bounds a disk D a embedded in D 2 . The group B a = A4(D a , P fl D a ) is a copy 
of the Artin braid group on \P fl D a \ strings which can be assumed to be embedded in 
B m = M(D 2 ,P) via the homomorphism M(D a , P n D a ) -> M(D 2 ,P) induced by the 
inclusion D a C -D 2 . It is a non-trivial but well known fact that the above homomorphism 
is injective. Such a subgroup is called geometric subgroup of B m . Assume a is generic. 
Rolfsen proved in [Ro] that the normalizer of B a in B m is equal to its commensurator 
in B m , and is equal to the stabilizer of [a]. He also proved that the action of B m in 
Xi(D 2 7 P) has noncommensurable stabilizers, infering by this way that the stabilizer of [a] 
is its own commensurator in B m . Some of these results have been generalized to parabolic 
subgroups of Artin groups [Pa] , to geometric subgroups of surface braid groups [PR1] , and 
to geometric subgroups of mapping class groups [PR2]. 

3. Constructing irreducible representations 

Let G be a countable discrete group. Define the commensurator of a subgroup H of 
G to be: 

Com.a{H) = {g e G ; gHg~ x and H are commensurable} . 

Theorem 3.1 (Mackey [Mac], Burger, de la Harpe [BH]). Let H be a subgroup ofG. The 
following statements are equivalent: 

(1) Com G (H)=H; 

(2) the left regular representation Xq/h of G in l 2 (G/H) is irreducible; 

(3) for all finite dimensional irreducible unitary representation tv of H, the induced 
representation Ind^7r is irreducible. 

Furthermore, if (l)-(3) are satisfied (i.e. ComGr(H) = H), then unitary induction provides 
a well defined and injective map 

Indg :H id ^G. □ 

Theorem 3.2 (Mackey [Mac], Burger, de la Harpe [BH]). Let Hi and H 2 be two subgroups 
of G such that Comer (Hi) = Hi and ComGr(H 2 ) = H 2 . The following statements are 
equivalent: 

(1) Hi and H 2 are conjugate; 

(2) the left regular representations Xq/ Hi and Xq/ H2 are equivalent; 

(3) there exist Gnite dimensional irreducible unitary representations tt\ and 7r 2 of Hi 
and H 2 , repectively such that Ind Hi 7Ti and Ind H2 7r 2 are equivalent. 

In particular, in case Hi and H 2 are not conjugate, then unitary induction provides a well 
defined and injective map 

Ind : Hf H H 2 d — > G . □ 
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Let G x X — > X be an action of G on some discrete set X. For x G X, let 

Stab(cc) = {g G G ; pec = x} 

denote the stabilizer of x. We say that the action has noncommensurable stabilizers if 
Stab(cc) and Stab(y) are not commensurable for all x, y G X, x 7^ y. 

Proposition 3.3 (Burger, de la Harpe [BH]). Let G x X — > X be an action with noncom- 
mensurable stabilizers. 

(i) ComG(Stab(cc)) = Stab(cc) for all x G X; 

(ii) Stab (a;) and Stab(y) are conjugate if and only if x and y are in the same orbit. □ 

Corollary 3.4 (Burger, de la Harpe [BH]). Let GxX — > X be an action with noncommen- 
surable stabilizers. Let O denote the set of orbits in X . Then unitary induction provides 
a well defined and injective map 

TT - — : fd 
Ind : ] [ Stab(or) -> G . □ 

xeo 



4. Curves and Dehn twists 

As pointed out in the introduction, the statements of this section are proved in [PR2] . 
So, we do not give the proofs here and refer to [PR2] for them. 

The index of intersection of two simple closed curves a, b : S 1 — > M \ P is: 

/(a, b) = inf-fja' n b'\ ; a' ~ a and b' ~ 6} . 

Note that: 

(1) if a is not generic, then /(a, 6) = for every simple closed curve b; 

(2) if a ~ b, then /(a, 6) = 0. 

Proposition 4.1. Let (ai, . . . , a r ) be a generic family of disjoint curves. For every i G 
{1, . . . , r}, there exists a simple closed curve b : S 1 — > M\P such that a,- fl 6 = for j 7^ i, 
and fl 6| = I{a^ b) > 0. □ 

Proposition 4.2. Let (ai, . . . , a r ) be a generic family of disjoint curves and b : S 1 — > M\P 
a simple closed curve such that /(a^, 6) = for aii i G {1, . . . , r}. Then there exists a simple 
closed curve b' : S 1 — > M\P isotopic with b and such that ai fl b' = for aii z G {1, . . . , r}. 
□ 
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Proposition 4.3. Let (a±, . . . , a r ) and (b±, ... , b r ) be two generic r -families of disjoint 
curves such that di ~ bi for all i G {1, . . . , r}. Then there exists an isotopy h t G Ti(M, P), 
t G [0, 1], such that ho =identity and fei o o, = bi for all i G {1, . . . , r}. □ 

By a subsurface N of M we mean a closed subset which is also a surface and for which 
we always assume dN n P = 0. We say furthermore that iV is essential if each component 
of M \ AT which is a disk has a nonempty intersection with the puncture set P. 

Proposition 4.4. Let N be an essential subsurface of M, and let a, 6 : S 1 — > iV \ iV n P 
be two generic closed curves. Assume that a is not isotopic with a boundary curve of N. 
Then a is isotopic with b in M\P if and only if a is isotopic with b in N \ N fl P. □ 

Let a : S 1 — > M \ P be a simple closed curve interior to M. Choose a tubular 
neighborhood w of a (namely, an oriented embedding v : S 1 x I M \ P such that 
v(z, 1/2) = a(z) for all 2 G and define the homeomorphism T a G H(M, P) by: 

(T a ov)(z,t)=v(e 2 ^ t z,t) , 

and T a is the identity on the exterior of the image of v in M. The Dehn twist along a is 
defined to be the element r a of AA(M, P) represented by T a . Note that: 

(1) the definition of r a does not depend on the choice of the tubular neighborhood; 

(2) r a = r b if a ~ 6; 

(3) r a = r a -i; 

(4) r a = 1 if a is not generic; 

(5) T a (a) = a; 

(6) let f G M{M,P), and let /i G H(M,P) which represents f; then ^r a ^ _1 is the 
Dehn twist along h(a); 

(7) r a and r b commute if /(a, 6) = 0. 

The following proposition is a special case of a formula in [FLP] . 
Proposition 4.5. Let a,b : S 1 ^ M \ P be two simple closed curves and n G Z. Then 

J(r^(6),6) = |n|/(a,6) 2 . □ 



The next two propositions can be also found in [IM] . 

Proposition 4.6. Let a 7 b : S 1 ^ M \ P be two generic closed curves and j, k G Z \ {0}. 
If t ^ = r b fc , then a ~ 6. □ 

Proposition 4.7. Let a,b : S 1 ^ M \ P be two generic closed curves and j, k G Z \ {0}. 
If and r b fc commute, then /(a, 6) = 0. □ 
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Proposition 4.8. Consider r generic closed curves ai, . . . , a r such that: 

(a) ai fl aj = 0, for i ^ j; 

(b) ai is neither isotopic to aj nor aj 1 , for i ^ j. 

Then the subgroup of M(M, P) generated by r ai , . . . , r ar is a free abelian group of rank r. 
□ 

Remark. The r-tuple (ai, . . . , a r ) of the above proposition in not necessarily a generic 
family of disjoint curves in the sence that may be isotopic with a boundary curve. 

5. The action 

Let a, P G X(M,P). According to the terminology on simplicial complexes, we say 
that a is a face of (3 if there exists a generic family of disjoint curves (a±, . . . , a r ) such that 
(3 = [ai, . . . , a r ] and a = [ai, . . . , a s ] for some s G {1, . . . , r}. 

Proposition 5.1. Let a, f3 G X(M, P) such that a is not a face of f3. Then the Stab(/3)- 
orbit of a is infinite. 

Proof. Write a = [a±, . . . , a r ] and (3 = [bi, . . . , b s ]. The hypothesis "a is not a face of /?" 
means that there is some ai which is neither isotopic with bj nor for all j £ {1, . . . , s}. 

Assume first that there exists some bj such that /(a^, bj) > 0. Consider the Dehn twist 
Tb j along bj. Let n G Z \ {0}. If t^ (q) = a, then r^_(ai) ~ a^ 1 for some k G {1, . . . , r}, 
thus 

I(r^(a l ),a l )=I(af,a l ) = 0. 
On the other hand, Proposition 4.5 implies 

I(T^(ai),ai) = \n\I(a t ,bj) 2 ^ . 

So, T£:( a ) 7^ a for all n G Z \ {0}. Since r^. G Stab(/9), it follows that the Stab(/?)-orbit of 
a is infinite. 

Assume now that I{ai,bj) = for all j G {l,...,s}. By Proposition 4.2, we may 
assume that Hbj = for all j G {1, . . . , s}, namely that (&i, . . . , b s , a^) is a generic family 
of disjoint curves. Proposition 4.1 provides a generic closed curve c such that bj D c = 
for all j G {l,...,s} and |a^ D c| = 7(aj,c) > 0. As before, r"(a) = a would imply 
I(r™(ai),ai) = 0, but Proposition 4.5 guarantees that I(r™(ai),ai) ^ 0, thus r^(a) ^ a 
for all n G Z \ {0}. Since r c G Stab(/?), it follows that the Stab (/?) -orbit of a is infinite. □ 

An action of a group G on a set X is called large if, for all x, y G X, x ^ y, the 
Stab(x)-orbit of y is infinite. It is shown in [BH] that large actions have noncommensurable 
stabilizers. Let a, f3 G X r (M, P), a//?. Then a cannot be a face of /3. So: 

Corollary 5.2. The action of M(M, P) on X r (M, P) is large for all r. □ 
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More generally: 

Corollary 5.3. The action of A4(M, P) on X(M, P) has noncommensurable stabilizers. 

Proof. Let oj, (3 G X(M, P), a ^ (3. We may assume that a is not a face of (3. Proposition 
5.1 provides an infinite sequence {gi}i>o in Stab(/3) such that g^a ^ gja for i ^ j. Then 

^(Stab(a) n Stab(/3)) ^ ^(Stab(a) n Stab(/?)) 

for i 7^ j, hence Stab(a) fl Stab(,9) has infinite index in Stab(/3). □ 

So, by Proposition 3.3: 

Corollary 5.4. Let a G X(M, P). Then Stab(a) is its own commensurator in M.{M, P). 
□ 

We turn now to the consequences of Proposition 5.1 on irreducible unitary represen- 
tations of the mapping class group. For a G X(M, P), we denote by A Q the left regular 
representation of M{M, P) on l 2 (M(M, P)/Stab(a)). 

Corollary 5.5. (i) Let a G X(M, P). Then A a is an irreducible unitary representation of 
M{M,P). 

(ii) Let a,(3<E X(M, P). The representations A Q and Xp are equivalent if and only if 
a and (3 are in the same A4(M, P)-orbit. In particular, A a and \p may not be equivalent 
if they have different ranks. □ 

Let O r denote the set of M(M, P) -orbits in X r (M, P). 

Corollary 5.6. Unitary induction provides a well defined and injective map 

Ind : II ( II Stab(a) fd ] -> M{M,P) . □ 

r=l \a£O r / 



A natural problem which arises from Corollaries 5.5 and 5.6 is to find a method to 
determine whether two classes a, (3 G X r (M, P) are in the same Ai(M, P) -orbit, and to 
find a representative for each .M(M, P)-orbit. We see now that this question has a natural 
answer in topological terms. 

Let A = (ai, . . . , a r ) be a generic r-family of disjoint curves. We denote by Ma the 
natural compactification of M\(Uj" =1 ai) and by pa '■ Ma — * M the continuous map induced 
by the inclusion map M \ (U^ =1 ai) — > M. By abuse of notation, we use P for Pa\P)- Let 
iV be a component of Ma and c : 5" 1 — > dN a boundary curve of N. We say that c is an 
exterior boundary curve of iV if pa ° c is a boundary curve of M. Let : 5 1 — > M \ P 
be one of the curves of A. There are two distinct boundary curves q, c' { : S 1 — > of 
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Ma such that pa° Q = Pa° c[ = a>i- Let iV and N' be the components of Ma having 
and as boundary curves, respectively. We say that aj is a separating limit curve of A" if 
N N' , and a« is a nonseparating limit curve of N if N = N' . 

We say that A determines a pantalon decomposition (or simply A is a pantalon decom- 
position) if, for each component A" of Ma, (N, P (1 N) is a pantalon (of any type). Such a 
family exists if and only if neither (M, P) is as in Example 1 or 2, nor (M, P) = (T 2 , 0). 
A generic family A = (ai, . . . , a r ) is a pantalon decomposition if and only if r is maximal 
(namely, X S (M,P) = for s > r and X r (M,P) ^ 0). This maximal r is given by the 
formula r = 3g + m + q — 3, and the number of pantalons is 2g + m + q — 2, where g is the 
genus of M, m = |P|, and (/ is the number of boundary components of M. 

The proof of the following proposition is left to the reader. 

Proposition 5.7. Assume neither (M, P) is as in Example 1 or 2, nor (M, P) = (T 2 , 0). 
Let g denote the genus of M , m = \P\, and q the number of boundary components of M. 

(i) Let A = (ai,...,a r ) and B — (bi, . . . ,b r ) be two generic r-families of disjoint 
curves. The class [A] and the class [B] are in the same M(M, P) -orbit if and only if there 
exist: 

• a one-to-one correspondence between the components of Ma and the components 
of M B} 

• a permutation a G S r , 

such that every pair (N, N') composed by a component N of Ma and its corresponding 
component N' of Mb satisfies the following properties: 

• the genus of N is equal to the genus of N', the number of boundary components of 
N is equal to the number of boundary components of N' , and |P D N\ = \P H N'\; 

• if c : S 1 — > dN is an exterior boundary curve of N , then there exists an exterior 
boundary curve c' : 5" 1 — > dN' of N' such that Pa°c = Pb°c'', 

• if aj is a separating limit curve of N , then b a ^ is a separating limit curve of N' ; 

• if Oi is a nonseparating limit curve of N, then b a ^ is a nonseparating limit curve of 

N'. 

(ii) Let A = (ai, . . . , a r ) be a generic family of disjoint curves. There exist generic 
closed curves a r+ i, . . . , a s (where s = 3g + m + q — 3) so that (ai, . . . , a r , a r+ i, . . . , a s ) is 
a generic family of disjoint curves which determines a pantalon decomposition. 

(in) Let s = 3g + m + q — 3. Let ti,t2,ts be three non-negative integers such that 
2ti+t2 = m andti+t2+t3 = 2g+m+q — 2. Let (M, P) denote the (non-connected) surface 
with punctures which is the disjoint union of t\ pantalons of type I, ti pantalons of type II, 
and ts pantalons of type III. Choose 2s distinct boundary curves ci, . . . , c s , c[, . . . , c' s of M 
(provided with the orientation induced by the one of M). We have (up to homeomorphism ) 
M = M I ~, where we identify the points of the curves Ci and c\ for all i G {1, . . . , s} by 
Ci(z) = (c' i )~ 1 (z). Denote the natural projection by p : M — > M. Then P = p(P). Write 
Oi = p o a for all i G {1, . . . , s}. Then A = (ai, . . . , a s ) is a pantalon decomposition of 
(M, P), and every M(M, P)-orbit of pantalon decompositions has a representative which 
can be obtained in this manner. □ 
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Observe that the above proposition shows that there are only finitely many M(M, P)- 
orbits of pantalon decompositions, and that every generic family of disjoint curves is a face 
of a pantalon decomposition, so: 

Corollary 5.8. There are only finitely many A4(M, P) -orbits in X(M, P). □ 

We turn now to see that the action of M.{M,P) on X{M,P) fournishes examples 
of pairs of self-commensurating subgroups, one included in the other, and that this phe- 
nomenon shows the limit of a possible generalization of Theorem 3.2. 

Choose a generic 2-family (a, b) of disjoint curves such that [a] and [b] are in different 
M.{M, P)-orbits; for example, take a = ai and b = a 4 in Example 4. Then 

Stab([a,6]) C Stab([a]) . 

So, we have the following. 

Proposition 5.9. There exists an example of a group G and two subgroups Hq, Hi of G 
such that: 

(a) ComG(-£Zo) = Hq and Coto.g(Hi) = Hi; 

(b) H c H i; 

(c) H is not conjugate to Hi. □ 

Assume we are in the situation of Proposition 5.9. Choose a finite dimensional irre- 
ducible unitary representation n of H . The induced representation tti = Ind^7r is an 
irreducible representation of Hi, since Couig(Hq) = H implies Com# 1 (Hq) = H . Now, 
the equality Ind^Ind^ = Ind H(j implies: 

Indg o 7r = Ind^TTi . 

This shows that every representation of G induced by a finite dimensional irreducible 
unitary representation of Hq is also induced by an (infinite dimensional) irreducible unitary 
representation of Hi. 



6. The structure of the stabilizers 

In order to fully apply Corollary 5.6, we would like to be able to exhibit finite di- 
mensional irreducible unitary representations of the stabilizers, different from the trivial 
ones. As pointed out in the introduction, there is no systematic procedure to produce 
(finite dimensional) irreducible unitary representations of a given group, exept for virtu- 
ally abelian groups. We turn now precisely to show that Stab ([A]) is virtually abelian if 
A = (ai, . . . , a r ) is a pantalon decomposition. 
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Define the pure mapping class group VA4(M, P) as the subgroup of M.{M,P) of 
isotopy classes of homeomorphisms h G Ti(M,P) which pointwise fixe P. Let Ep denote 
the group of permutations of P. We have the exact sequence 



1 -> VM(M, P) -> M(M, P) -> E P -> 1 . 



Notice that the pure mapping class group of a pantalon (of any type) is the free abelian 
group generated by the Dehn twists along its boundary components. 

Define the cubic group Cub r to be the group of linear transformations / G GL(H r ) 
such that /(ej) G {±ei, . . . , ±e r } for all i G {l,...,r}, where {ei,...,e r } denotes the 
canonical basis of R r . There is a natural homomorphism Fa : Stab ([A]) — > Cub r defined 
as follows. Let £ G Stab ([A]), and let ft G H(M, P) which represents £. Then we set 



if ft(aj) 
if ft(ai) ~ a 



This homomorphism is not surjective in general. Its kernel is composed by the isotopy 
classes of homeomorphisms h G 7i(M, P) such that h{ai) ~ cxj for alH G {1, . . . , r}. 

Let ci, . . . , Cq be the boundary components of M. For a = [ai, . . . , a r ] G X(M, P), we 
denote by T(a) the subgroup of A4(M, P) generated by r 0l , . . . , r ar , r Cl , . . . , r c . Recall 
from Proposition 4.8 that this group is a free abelian group of rank r + q. Notice also that 
T(a) C Stab(a). 



Proposition 6.1. Let A = (ai,...,a r ) be a generic family of disjoint curves which 
determines a pantalon decomposition. Then 



T([A}) = VM{M, P) n KerP4 . 



Remark. Proposition 6.1 implies that T(L4]) has finite index in Stab ([A]) since both 
VM(M, P) n Stab(L4]) and KerF A have finite index in Stab(L4]). 

Proof. The inclusion T([A]) C VM(M, P) flKerP^ being obvious, it suffices to prove the 
reverse inclusion: VM{M,P) n KerP A C T([A]). Let £ G VM{M,P) n KerP A , and let 
h G Ti,(M, P) which represents £. We have hoai ~ for all % G {1, . . . , r}. By Proposition 
4.3, we may assume that /i o a t = a t for all % G {1, . . . , r}. By the structure of the pure 
mapping class groups of the pantalons, it follows that £ can be expressed as 

C — _ni n r mi m q 

•» ' Ol * * O r Cl * * Cg ' 

thus £ G T(L4]). □ 

We turn now to show that a given stabilizer can be expressed by means of mapping 
class groups of "smaller" surfaces. 
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Let A = (cxi, . . . ,a r ) be a generic family of disjoint curves. Consider the (non- 
connected) surface Ma of Section 5 which is the natural compactification of M\ (U^ =1 aj), 
and denote by iV~i, . . . , Ni the components of Ma- We write 

M(M A ,P) = M(N 1 ,N 1 nP) x ... x M(Ni,N t nP) , 
VM(M A ,P) = VM(N 1 ,N 1 nP) x ... x VM{N h Ni nP) . 

We have the exact sequence 

1 -> VM(M A , P) -> M(Ma-, P) -> E^np x ... x E^ nP -> 1 . 

Recall also that p/i : — > M denotes the continuous map induced by the inclusion map 
M \ (U[ =1 a;) -> M. This map induces a homomorphism p* : M{M A ,P) -> .M(M,P). 
Notice that, by Proposition 4.3, 

Imp* = KerP^ C Stab(L4]) . 

Lemma 6.2. Assume neither (M, P) is as in Example 1 or 2, nor (M, P) = (T 2 ,0). 
Let A = (ai, . . . , a r ) be a generic family of disjoint curves. For i G {1, . . . , r}, let q and 
denote the boundary curves of Ma such that pa ° c% = Pa ° c[ = ai. Then Kerp* is 
generated by {t^tJ 1 , . . . , t^t^ 1 } and is a free abelian group of rank r. 

Proof. Let T denote the subgroup of M.(Ma, P) generated by {t Ci t7 , • • • , r c rT 1 }. The 
fact that T is a free abelian group of rank r is a direct consequence of Proposition 4.8, and 
the inclusion T C Kerp* is obvious. So, it remains to prove the inclusion Kerp* C T. 
Let £ G Kerp*. Observe first that we have the commutative diagram: 

1 -> VM(M A ,P) -> M(M i? P) -> Ejv inP x . . . x Ejv in p -> 1 
1 -> VM(M,P) -> M(M,P) -> E P -> 1 

and that the homomorphism Ejv in pX...xSjv |n p — > Ep is injective, thus £ G P.M(Ma, P). 

Let ft : Ma — > Ma be a homeomorphism which represents £. We have ft o q = Cj 
and ft o = c- for all z G {1, . . . , r}. Let di, . . . , d q be the boundary curves of M, and 
let ei,...,e ? be the corresponding boundary curves of Ma (i.e. ej satisfies pa ^ = 
di). We also have ft o ej = e^. We choose essential closed curves a r +i, . . . , a 8 such that 
(ai, . . . , a r , a r +i, . . . , a s ) is a pantalon decomposition of (M, P). For j G {r + 1, . . . , s}, let 
Cj denote the essential closed curve of Ma such that pA°Cj = Notice that (c r+ i, . . . , c s ) 
determines a pantalon decomposition of (M A , P), and ci, . . . , c r , c' l7 . . . , c' r , ei, . . . , e 3 are 
the boundary components of M A . Let j G {r + 1, . . . , s}. Since £ G Kerp*, pa h o Cj ~ 
Pa o Cj. By Proposition 4.4, it follows that ft o Cj ~ Cj. So, by Proposition 4.3, we may 
assume ho Cj = Cj. By the structure of the pure mapping class groups of the pantalons, it 
follows that £ can be written 

t — _wi u r v x v r u r+1 u s (Oi w 9 

S 'ci • • • 'c r 1 c[ ' ' ' < 'c r+ i • • • 'c s 'ei • • • 'e 9 5 
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where u±, . . . ,u s ,vi, . . . , v r , w\, . . . , w q e Z. Finally, the equality 



1 — f J *V<i) — '01 • • • 'a r 'o r+ i • • • 'o s 'di • • • 'd g 

implies by Proposition 4.8: 

Ul + Vi = . . . = U r + V r = U r +l = . . . = U s = Wi = . . . = W q = , 

thus 

e=(r Cl T c 7 1 r...(r Cr r c 7 1 r • □ 

The above considerations lead to the following proposition. 

Proposition 6.3. Assume neither (M, P) is as in Example 1 or 2, nor (M, P) = (T 2 , 0). 
Let A = (ai,...,a r ) be a generic family of disjoint curves. Then we have the exact 
sequence 

1 -> Z r -> 7W(M A , P) -> Stab([A]) -> Cub r . □ 
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